Measuring Dependence Powerfully and Equitably

For high-dimensional data sets, it is common to evaluate a measure of dependence onevery variable pair and retain the highest-scoring pairs for follow-up. If the statistic usedsystematically assigns higher scores to some relationship types (e.g., linear, exponential,etc.) over others, important relationships may be overlooked because of their type. Thisdifficulty is avoided if the statistic is equitable [1], i.e., if, for some measure of noise, itassigns similar scores to equally noisy relationships regardless of relationship type.In this paper, we introduce and characterize a population measure of dependence calledMIC∗. We show three ways that MIC∗ can be viewed: as the population value of MIC, ahighly equitable statistic from [2]; as a canonical “smoothing” of mutual information; and asthe supremum of an infinite sequence defined in terms of optimal one-dimensional partitionsof the marginals of the joint distribution in question. Based on this theory, we introducean efficient algorithm for computing MIC∗ from the density of a pair of random variables,and we define a new consistent estimator MICe for MIC∗ that is efficiently computable.(In contrast, there is no known polynomial-time algorithm for computing MIC.) We showthrough simulations that MICe has better bias-variance properties than MIC, and that ithas high equitability with respect to R on a set of functional relationships.Traditional data exploration focuses also on the power of using a statistic to test a nullhypothesis of statistical independence. While MICe is designed for equitability rather thanindependence testing, we introduce a related statistic, TICe, that is a trivial side-productof the computation of MICe. We prove the consistency of independence testing based onTICe and show in simulations that this approach achieves excellent power.This paper is accompanied by a companion paper [3] focused on in-depth empirical eval-uation of several leading measures of dependence. That paper shows that MICe and TICeachieve state of the art equitability with respect to R and power against independence,respectively. Taken together, our results show that MICe and TICe are a valuable new pairof tools for exploratory data analysis.